Some systems offer to their customers services, e.g., in the form of callable components. Such systems are not necessarily established by one party alone. For example, in many cases, one party establishes the framework and allows other parties to deploy components into this framework that are callable by the customers or otherwise pertain to the system. This multi-ownership property may or may not be transparent to a given customer. Such systems might additionally or alternatively enable parties owning the components deployed to the platform to be able to more or less be constantly capable of providing, updating, deleting, or otherwise modifying, “their” components.
Apart from making the systems change their behaviors in potentially unexpected ways, the ability for parties to change their deployed components also enables the systems to perform tasks using possibly unexpected combinations of components that may or may not have been possible and/or intended by any of the component suppliers. Systems that behave as such are often referred to as “emergent.”
For convenience, the party providing the framework may be referred to as a “provider” and the parties adding components to this framework may be referred to as “partners.” An example involves a provider offering a platform for scheduling holidays, with the partners providing components for ordering services in this scenario. Those services could include, for example, flights, hotels, sightseeing-trips, and/or the like.
The provider of an emergent framework oftentimes will have an interest that the components included by partners work satisfactorily. Thus, it may be advisable for a provider to ask the partners to also provide a test relevant to each component deployed to the framework. The provider thus may be able to run these tests (e.g., at regular intervals) to help ensure that each component works as expected, on its own. When and how often such tests run may depend, for example, on the frequency they are called, the relevance for the provider and/or the partner, and/or other factors.
The nature of an emergent system as described above provides value because it enables spontaneous collaboration between components provided by different partners. It will be appreciated, then, that the provider therefore has a high interest in not only regularly testing the components themselves, but also in testing those collaborations. But because neither the provider nor the partners can always foresee which combinations of components will exist during the platform's lifecycle, suitable tests for testing those combinations cannot always be provided by the provider or by the partners involved.
Thus, it will be appreciated that there is a need in the art for techniques for improving testing in connection with components themselves, and with collaborations as between multiple components, provided in emergent systems and/or the like.
One aspect of certain example embodiments relates to techniques that allow for testing mechanisms that are as emergent as the platform's service offerings.
Another aspect of certain example embodiments relates to the ability to automatically test on emergent platforms: single components as provided by partners, often-used combinations of components provided by possibly different partners, combinations of components provided by possibly different partners in which the components are each similar to corresponding components in such combinations, and/or specific combinations using additional knowledge acquired by combining asset's semantics and information obtained by system monitoring.
Another aspect of certain example embodiments relates to applying semantics to combination testing for combinations of services and/or components that can only normally be detected dynamically (and potentially cannot be predicted with a high degree of confidence).
Still another aspect of certain example embodiments relates to the detection and monitoring of combinations together with the automatic running of combination tests, e.g., for similar combinations as specified by a user and/or as detected using semantic concepts.
In certain example embodiments, a method for testing services deployed to hardware components in an emergent system is provided. The method comprises detecting when multiple calls to one or more of the services occur within (a) a single semantic context and/or (b) a predefined time period. Combinations of called services are identified based at least in part on said detecting. A registry entry is created, in a registry stored on a non-transitory computer readable medium, for each said identified combination, with each said registry entry associating the respective identified combination with registered assets representing the called services of the respective identified combination. A determination is made as to whether an identified combination is test-worthy. At least one executable test is associated, via at least one processor, with each said identified combination that is determined to be test-worthy.
In certain example embodiments, a method for testing services deployed to hardware components in an emergent system is provided. A registry stored on a non-transitory computer readable medium is maintained. The registry includes entries for combinations of services that are called in connection with one another, with each said entry associating the respective combination with registered assets representing the called services of the respective combination and indicating whether the respective combination is test-worthy. At least one executable test associated with a given combination is automatically executed, via at least one processor, (a) when the given combination has a service associated therewith that has been updated, and/or (b) at regular intervals.
In certain example embodiments, a computer system for testing software and/or hardware components provided by in an emergent system is provided. Processing resources include at least one processor and a memory. A registry is stored on a non-transitory computer readable medium. The processing resources are configured to cooperate to at least: detect when multiple calls affecting one or more of the components occur; identify combinations of components based at least in part on said detection; create a registry entry in the registry for each said identified combination, with each said registry entry associating the respective identified combination with registered assets representing the components of the respective identified combination; determine whether an identified combination is test-worthy; and associate at least one executable test with each said identified combination that is determined to be test-worthy.
In certain example embodiments, there is provided a non-transitory computer readable storage medium tangibly storing instructions that, when executed by at least one processor of a system, perform a method as described herein.
These aspects, features, and example embodiments may be used separately and/or applied in various combinations to achieve yet further embodiments of this invention.